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Device and method tor sensing data input 


(57) 


A user input device comprising: a sensor capa- 


sor for detecting movement of skin across the sensor; 



ble of sensing surface relief of human skin; a movement 
detector responsive to variation of the output of the sen- 



and user interface apparatus responsive to the move- 
ment detector to alter Its state In correspondence to the 
detected movement. 
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Description 

[0001 ] This invention relates to sensing data input, es- 
pecially sensing input of movennent and/or pointing data 
by means of a sensor such as a fingerprint sensor. 
[0002] One important way of making electronic equip- 
ment easier to use is to make operation of the equip- 
ment's user interface more intuitive. To this end, many 
electronic devices have graphical and/or menu-driven 
user interfaces. In those user interfaces options are dis- 
played on a screen and can be scanned through or se- 
lected on the basis of the movement of a pointer. The 
pointer may be a cursor which is displayed, or may be 
a virtual pointer which is not displayed but can be con- 
sidered by a user to be conceptually linked to the oper- 
ation of the display (for example in the scrolling of a 
menu list). Input means is then provided for detecting 
inputs from a user that can cause movement of the 
pointer. Such input means preferably detect physical 
movement of part of the user's body, whereupon the 
pointer can be caused to move in a corresponding way. 
A number of types of such input means are currently 
preferred. For detecting motion In one dimension there 
are rotatable wlieels or rollers linked to rotation sensors. 
For detecting motion in two dimensions there are mice 
and trackballs including multiple rotation sensors, track- 
points having multiple force sensors, joysticks having 
multiple sensors for detecting angular deflection, and 
touchpads which can determine the location of a pres- 
sure source such as a fingertip within the boundaries of 
the touchpad and over time establish the motion of the 
pressure source within those boundaries. An array of 
buttons or keys, suitably marked with direction arrows, 
can also be used. 

[0003] These forms of motion detection all have dis- 
advantages, especially for highly portable equipment 
such as mobile telephones. Where equipment Is intend- 
ed to be highly portable it is greatly desirable that it Is 
small and light-weight and, to improve reliability, that it 
has no moving parts. Wheels, rollers, trackballs, mice 
and joysticks must be large enough to be conveniently 
engaged by a user's hand and have moving parts and 
rotation or other detectors that are easily disrupted by 
dirt that can accumulate during use. Dirt is a particular 
risk for items such as mobile telephones that are carried 
continually and often exposed to dusty or dirty environ- 
ments. Trackpoints are highly compact but are difficult 
for some users to operate because the user's finger en- 
gaged on the trackpoint does not change position sig- 
nificantly during use. Touchpads can be found to be eas- 
ier to operate because the user's finger must move 
across the surface of the touchpad in order to provide 
an input. Touchpads have therefore become popular for 
large-sized items of portable equipment such as laptop 
computers. However, touchpads are generally unsuita- 
ble for smaller devices such as modem mobile tele- 
phones because of the space needed to provide room 
on the touchpad for a finger to be able to be moved with- 



in its boundaries. With a touchpad, sensed movement 
is dependant principally on the location of pressure on 
the pad. Buttons and keys occupy a relatively large 
amount of space on a device and are less intuitive for 

5 users to operate than are other more tactile input 
means. Also, joysticks, mice, rollers and rollerballs are 
not capable of conveniently sensing rotating motion. 
[0004] There is therefore a need for an improved 
means for control of user interfaces. 

10 [0005] According to one aspect of the present inven- 
tion there is provided a user input device comprising: a 
sensor capable of sensing surface relief of human skin; 
a movement detector responsive to variation of the out- 
put of the sensor for detecting movement of skin across 

15 the sensor; and user Interface apparatus responsive to 
the movement detector to alter its state in correspond- 
ence to the detected movement. 
[0006] According to another aspect of the present in- 
vention there is provided a method of operation in a user 

20 input device comprising a sensor capable of sensing 
surface relief of human skin, the method comprising: de- 
tecting movement of skin over the sensor; and altering 
the state of user interface apparatus in response to the 
detected movement. 

25 [0007] The sensor suitably comprises a plurality of 
sensing units. The sensing units are preferably ar- 
ranged In a substantially planar array. The sensing units 
may be optical sensing units or sensing units of another 
type. The sensor may be a fingerprint sensor. The sen- 

30 sor is preferably capable of detecting surface relief of a 
depth of 1 or 2 jxm or less. 

[0008] The movement detector is preferably respon- 
sive to variation of the output of the sensor for detemnin- 
Ing the direction of movement of skin over the sensor 
35 Most preferably the sensor comprises a first sensing 
section extending In a first direction and a second sens- 
ing section extending in a second direction. One or both 
of the sensing sections may comprise a linear arrange- 
ment of sensing units. The first and second directions 
40 are preferably orthogonal. The movement detector is 
preferably operable to compare successive outputs 
from the first sensing section to detect movement in the 
first direction and to compare successive outputs from 
the second sensing section to detect movement in the 
45 second direction. The movement detector is preferably 
operable to determine a composite direction of move- 
ment over the sensor from combination of detected 
movement In the first and second directions. The move- 
ment detector is preferably operable to detect move- 
so ment by detecting changes overtime In the relief detect- 
ed by parts of the sensor. 

[0009] The device suitably comprises storage appa- 
ratus capable of storing successive outputs from the 
sensor in a memory so as to form a record of the surface 
55 relief of skin drawn over the sensor. That record may 
represent a fingerprint scan. 

[0010] The user interface apparatus may include a 
display and a display controller capable of controlling 



2 



BNSDOCID: <EP 1 1 1338SA2l.L> 



3 



EP1 113 385 A2 



4 



the display so as to cause at least part of an Image to 
move over the display in correspondence to detected 
movement over the sensor. The said at least part of an 
Image may be a cursor, and icon or one or more menu 
options. The display controller may be capable of con- 
trolling the display so as to cause the said at least part 
of an image to move over the display in apparently the 
same direction as the detected movement over the sen- 
sor. For example, if the sensor is in substantially the 
same plane as the display then those directions of 
movement may be substantially the same. 
[001 1] The present invention will now be described by 
way of example with reference to the accompanying 
drawings, in which 

figure 1 shows schematically the architecture of a 
communication terminal; 
figure 2 illustrates a fingerprint sensor; 
figure 3 illustrates modes of operation of a user In- 
terface of the mobile telephone of figure 1 ; 
figure 4 Is a cross-section of a fingerprint sensor; 
and 

figure 5 Is a flow diagram showing one example of 
the operation of the user Interface of the mobile tel- 
ephone of figure 1 ; 

[0012] The communication terminal of figure 1 in- 
cludes a fingerprint sensor 5 which is capable of sensing 
the surface relief of an object over it. The fingerprint sen- 
sor is connected to detector circuitry 12 which is capable 
of detecting the direction of motion of an object across 
the fingerprint sensor by comparing two or more suc- 
cessive outputs from the sensor. The detector can gen- 
erate a signal at 13 indicative of the detected direction 
of motion. That signal is input to control processor 3 of 
the terminal. Control processors then interprets that sig- 
nal appropriately, as a user input. For example, the con- 
trol processor 3 may cause motion of a cursor on the 
display in a corresponding direction to the detected 
movement. 

[0013] The mobile terminal of figure 1 comprises an 
antenna 1 connected to a radio interface 2 for transmis- 
sion and reception of signals from a radio network. A 
control processor 3 is connected to the radio interface 
whereby the control processor may provide the radio in- 
terface with data for transmission to the network, and 
may receive data from the network via the radio inter- 
face. The control processor is connected to input and 
output apparatus of the tenninal, including keypad 4, fin- 
gerprint sensor 5, microphone 6. loudspeaker 7 and vis- 
ual display 8. The control processor is capable of receiv- 
ing Inputs from the input devices 4, 5 and 6 and providing 
outputs to the output devices 7 and 8. 
[001 4] The fingerprint sensor 5 is shown in more detail 
in figure 2. The sensor comprises a primary linear array 
10 of around 256 optical or contact sensors (e.g. sen- 
sors 11) located closely together, and a secondary linear 
array 11 of around 20 less closely spaced sensors dis- 



posed orthogonally to the primary ari;ay. Each sensor is 
connected to detector circuitry shown schematically at 

12 which can generate an output signal at 13. Output 

13 is connected to the control processors. The primary 
5 array iscapable of sensing the detailed shape of a linear 

section of an object In contact with It by means of the 
level of light incident on, the degree of contact with, the 
force on, temperature at or conductivity to each of the 
sensors of the primary array. The control circuitry is ar- 

10 ranged to periodically scan the sensors to receive their 
outputs and thereby periodically generate an output sig- 
nal indicative of the suriace relief pattern sensed by the 
primary array. The outputs from the sensors may be 
sensed digitally as one of a finite number of levels - for 

IS example 2, 4, 8 or 16 levels. Successive signals gener- 
ated in this way can be stored - so as to build up an 
Image of the shape of an object passing over the sen- 
sors - or compared, along with data from the secondary 
array - so as to determine the direction of movement in 

20 the plane of the arrays of an object passing over the sen- 
sors. Data resulting from the primary array and the sec- 
ondary array are included in the output signal at 13. 
[0015] The sensor may be capable of detecting sur- 
face relief of an object lying over it and/or that is swept 

25 over it. 

[0016] The fingerprint sensor may be the FingerChip 
FC15A140 available from Thomson-CSF. Other de- 
signs of fingerprint sensor may be used. 
[0017] In one mode of operation a user can slide his 

30 fingertip over the primary arriay as illustrated at 1 4 in fig- 
ure 2. The secondary array allows the sliding motion of 
the user's finger over the array to be detected when suc- 
cessive outputs from the secondary array are com- 
pared. By storing successive signals from the primary 

35 array as the user's finger slides over the fingerprint sen- 
sor, the control processor can build up an image of the 
user's fingerprint. That image can be stored by the con- 
trol processor 3 in a memory 9. The stored image can 
then be checked against subsequent scans for use in 

40 authentication of operations of the terminal for security 
purposes. The period between successive scans may 
be fixed at around 1 .5ms, for example, or may be de- 
pendant on the rate of movement detected by the sec- 
ondary array as detennined at detector circuitry 12. The 
same principle may be used to scan the surfaces of oth- 
er objects than fingertips. In this mode of operation the 
signal at 1 3 that is output by the detector circuitry com- 
prises information defining the scanned image. 
[0018] In another mode of operation the fingerprint 

50 sensor of the mobile terminal of figure 1 can be used in 
conjunction with detector circuitry 12 and control proc- 
essor 3 to detect motion of a user's fingertip across the 
sensor. That motion may be used to help control the user 
interface of the phone. Successive outputs of the prima- 

55 ry and secondary array are taken frequently and com- 
pared by detector circuitry 12. The period between suc- 
cessive samples of the arrays may be, for example, 2ms 
or less. For greater accuracy rnore than two successive 
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output scans may be compared. Detector circuitry 12 
stores successive outputs. Detector circuitry 12 then 
analyses successive outputs witli the aim of Identifying 
similar features In them. The distance by which such 
features have moved between the scans in the axis of 
the primary and secondary arrays can then be deter- 
mined. Those distances can be combined as vectors to 
determine the composite direction of motion of the fin- 
gertip across the sensor. The rate of movement can also 
be determined using detector circuitry 12's [knowledge 
of the time between successive scans. The same prin- 
ciple can be used to determine the direction of move- 
ment over the sensors of other objects than fingertips, 
in this mode of operation the signal at 13 that is output 
by the detector circuitry comprises information defining 
the direction and rate of movement of the object over 
the sensor. 

[001 9] The required mode of operation Is indicated to 
the detector circuitry 12 by control processors over con- 
nection 13a. 

[0020] The fingerprint sensor is, of course, capable of 
detecting surface relief of other body parts or other ob- 
jects than fingertips. The fingerprint sensor is preferably 
capable of sensing surface relief digitally. The fingerprint 
sensor is preferably capable of sensing surface relief at 
a resolution of less than 0.1mm, more preferably less 
then 0.05mm and most preferably less than 0.01mm. A 
preferred sensor pitch is around SOjim. 
[0021] Figure 3 Illustrates some modes of operation 
of the user interface of the mobile terminal. 

• InflgureSadlsplay 8 of the terminal shows a column 
listing a number of options 20. A cursor21 highlights 
one of the options. In order to select one of the op- 
tions a user applies inputs to the terminal to cause 
the cursor to move up or down to another option, or 
to cause the options to move up and down through 
the cursor, until the desired option is highlighted. 
The user then applies an input to the tenninal to in- 
dicate that the desired option has been highlighted. 
A similar principle can be used with left/right motion 
to offer the user a row of options or a two dimen- 
sional array of options. 

• In figure 3b display 8 of the terminal shows a cursor 
25 and icons 26 distributed in two dimensions Each 
icon represents an option. In order to select one of 
the options a user applies inputs to the terminal to 
cause the cursor to move up, down, letter right until 
it lies overthe icon corresponding to the desired op- 
tion. The user then applies an input to the terminal 
to Indicate that the desired option has been high- 
lighted. 

• In figure 3c display 8 of the temiinai shows two op- 
tions 29: one on the right and the other on the left 
of the display. The user makes a single Input Indi- 
cating which of those options Is desired. 

[0022] In the mobile tenninal of figure 1 , the fingerprint 



sensor can be used to help the user to select one of the 
displayed options. When the user is to select an option 
the control processors Is arranged to show the appro- 
priate image on the display of the mobile terminal. The 

5 control processor then signals the detector circuitry 12 
to indicate that the circuitry should operate to interpret 
data from the fingerprint sensor so as to detect motion 
over the sensor. When such motion is detected by the 
detector circuitry 1 2 it is signalled to the control proces- 

10 sor 3 to allow the user interface to be controlled in a 
corresponding manner. For example, in the situation of 
figure 3a, up and down motion across the fingerprint 
sensor (as Illustrated at 32 and 33) may cause the op- 
tions to scroll correspondingly up or down or the cursor 

15 to move correspondingly up or down. In the situation of 
figure 3b, motion across the fingerprint sensor may 
cause corresponding motion of the cursor (as Illustrated 
at 36 and 37). in the situation of figure 3c, leftward mo- 
tion across the fingerprint sensor may cause the left op- 

20 tion to be selected and rightward motion across the fin- 
gerprint sensor may cause the right option to be select- 
ed. Other arrangements may, of course, be used. In or- 
der to Improve the usability of the user Interface it Is pre- 
ferred that the motion of parts of the image on the display 

25 con-esponds to the motion detected by the motion sen- 
sor, for example by appearing to a user to be the same 
direction as or othenvlse intuitively liked to the detected 
motion. 

[0023] In some user interface arrangements, for ex- 

30 ample those of figures 3a and 3b, in addition to making 
inputs to indicate motion a user must also make an input 
to indicate selection of an option. One way to permit this 
is my means of input from a selection button or key on 
the mobile terminal. Such a button may be part of a nor- 

35 mal keypad of the tenninal and separate from the fin- 
gerprint sensor Another way to permit this Is by means 
of input, for example a sudden input such as a finger 
tap, to the fingerprint sensor itself. This may be detected 
by the fingerprint sensor or by another input device at- 

40 tached to the fingerprint sensor. To detect such input by 
means of the fingerprint sensor Itself the detector cir- 
cuitry 12 may be arranged to sense a sudden change in 
the output from the sensor indicative of the sudden ap- 
plication of an object over the sensor and in response 

45 thereto to provide a selection signal to the control proc- 
essor 3. To detect such Input by use of another input 
means the arrangement in figure 4 may be used. In fig- 
ure 4 the fingerprint sensor 40 is mounted on a push- 
switch 41 . Sensor 40 and switch 41 provide inputs to 

50 detector circuitry 12. When fingerprint sensor 40 is 
tapped the switch 41 is operated, and in response there- 
to detector circuitry 1 2 provides a selection signal to the 
control processor 3, Switch 41 preferably provides an 
audible and/or tactile feedback to a user - for example 

55 by means of domed membrane 42 which can deform 
suddenly to the configuration shown at 43 when 
pressed. 

[0024] Figure 5 illustrates an example of the operation 
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of the mobile terminal of figure 1 . At step 50 a user turns 
on the terminal using power key 33 (figure 1). The 
processing equipment of the tenminal Is booted up (step 
51). The control processor 3 displays a log-in screen on 
display 8 and signals to the detector circuitry 12 that it 
should operate in fingerprint input mode (step 52). The 
user swipes his fingertip over the fingerprint sensor 5. 
Detector circuitry 12 fomns an Image of the user's fin- 
gerprint from combination of successive signals from 
the fingerprint sensor and transmits that image to control 
processor 3 (step 53). Control processor 3 compares 
that image to a stored Image of the authorised user's 
fingerprint (step 54) and if there is sufficient match al- 
lows access to further functions of the tenninal (step 55). 
OthenA^ise, further access is denied (step 56). The con- 
trol processor then displays a cursor and icons repre- 
senting options distributed over display 8 as illustrated 
in figure 3b and signals to detector circuitry 12 that it 
should operate in motion input mode (step 57). The user 
moves his finger over the fingerprint sensor in order to 
move the cursor (by means of detector circuitry 12 and 
control processor 3) correspondingly until it lies over a 
desired icon (step 58). The user then lifts his finger off 
the fingerprint sensor and taps it briefly to indicate se- 
lection of that icon, and as a result detector circuitry 
sends a selection signal to control processor 3 which 
performs the action associated with that icon (step 59). 
This process can continue in order to operate the func- 
tions of the temninal. 

[0025] Movement detected by the fingerprint sensor 
may be interpreted by the control processor in a way 
that does not cause corresponding movement of an item 
on the display. For example, up and down motion over 
the fingerprint sensor during a terminal call could cause 
the earpiece volume of the tenninal to increase or de- 
crease respectively. 

[0026] The fingerprint sensor may be supplemented 
by other means of user input to the control processor. 
Additional input means could include a keypad, for in- 
stance including numeric keys for dialling purposes, 
and/or a microphone for receiving voice Input from a us- 
er. The voice input could be for input of speech during 
voice tenninal calls and/or for voice activation of options 
if the mobile tenninal Is equipped with voice recognition 
apparatus. 

[0027] The operating methods and apparatus de- 
scribed above can be applied to equipment other than 
mobile temnlnals. 

[0028] One notable advantage of the input arrange- 
ment described above is that because the fingerprint 
sensor is capable of sensing the relief of an object (es- 
pecially the small scale surface relief of an object) it al- 
lows for an especially compact motion input device. The 
above input arrangement can be contrasted in particular 
to a touchpad. A touchpad is capable of sensing motion 
of an object only within the boundaries of the touchpad; 
therefore the touchpad must be large enough to accom- 
modate such movement. In the above arrangement, be- 



cause the surface relief of an object moving across the 
sensor can be sensed, the sensor can be of compara- 
tively small size. Thus unlike with a touchpad the input 
arrangement of the present invention is not limited to 

5 providing motion detection principally in dependence on 
the location of an object on the sensor - in contrast, mo- 
tion detection Is essentially independent of the location 
of an object on the sensor, and dependant on the de- 
tected motion of its surface relief across the sensor. The 

10 present invention is therefore particularly advantageous 
when applied to equipment that is of small size, such as 
user portable equipment. 

[0029] Another notable advantage of the above input 
arrangement is that the sensor may be used for reading 
15 of fingerprints (or other body parts) as well as sensing 
motion. The sensor and its associated circuitry may 
therefore have a security function as well as a more con- 
ventional input function. The present Invention Is there- 
fore especially advantageous when applied to equip- 
ment that is preferred to have a degree of security, when 
the fingerprint sensor can provide a means of allowing 
relative security of access to the functions of the equip- 
ment. Such equipment may be equipment that Is capa- 
ble of making a debit to a user's account and/or equip- 
ment that provides access to confidential information. 
Examples of equipment to which the present Invention 
may be applied include cameras, person computers, 
personal organisers, cash dispensers and door entry 
systems. 

[0030] Another point to note is that the above input 
arrangement is capable of sensing a rotating motion of 
a finger or the like over the sensor. This could usefully 
provide convenient data input, especially in conjunction 
with a corresponding visual display, for example to sim- 
ulate tuming of a key in a lock as the user's finger rotates 
substantially about a point. 

[0031] The applicant draws attention to the fact that 
the present invention may include any feature or com- 
bination of features disclosed herein either implicitly or 
explicitly or any generalisation thereof , without limitation 
to the scope of any of the present claims. In view of the 
foregoing description it will be evident to a person skilled 
in the art that various modifications may be made within 
the scope of the invention. 



Claims 

1 . A user input device comprising: 

a sensor capable of sensing surface relief of hu- 
man skin; 

a movement detector responsive to variation of 
the output of the sensor for detecting move- 
ment of skin over the sensor; and 
user interface apparatus responsive to the 
movement detector to alter its state in corre- 
spondence to the detected movement. 
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2. A device as claimed in claim 1 or2, wherein the sen- 
sor comprises a plurality of sensing units arranged 
in a substantially planar array. 

3. A device as claimed in claim 2, wherein the sensing 
units are optical sensing units. 

4. A device as claimed in any preceding claim, where- 
in the sensor is a fingerprint sensor. 

5. A device as claimed in any preceding claim, where- 
in the movement detector is responsive to variation 
of the output of the sensor for determining the di- 
rection of movement of skin across the sensor. 

6. A device as claimed in any preceding claim, where- 
in the sensor comprises a first sensing section ex- 
tending in a first direction and a second sensing 
section extending In a second direction, and the 
movement detector is operable to compare succes- 
sive outputs from the first sensing section to detect 
movement In the first direction and to compare suc- 
cessive outputs from the second sensing section to 
detect movement in the second direction. 

7. A device as claimed in claim 6, wherein the move- 
ment detector is operable to determine a composite 
direction of movement across the sensor from com- 
bination of detected movement in the first and sec- 
ond directions. 

8. A device as claimed in any preceding claim, Includ- 
ing storage apparatus capable of storing succes- 
sive outputs from the sensor in a memory so as to 
fomn a record of the surface relief of skin drawn 
across the sensor. 

9. A device as claimed in any preceding claim, where- 
in the user interface apparatus includes a display 
and a display controller capable of controlling the 
display so as to cause at least part of an Image to 
move across the display in correspondence to de- 
tected movement across the sensor 

10. A device as claimed In claim 9, wherein the display 
controller is capable of controlling the display so as 
to cause the said at least part of an image to move 
across the display In apparently the same direction 
as the detected movement across the sensor. 

1 1 . A device as claimed in any preceding claim, where- 
in the sensor is capable of sensing relative rotation 
of skin and the sensor. 

12. A user input device substantially as herein de- 
scribed with reference to the accompanying draw- 
ings. 



13. A communication tenninal comprising a user Input 
device as claimed in any preceding claim. 

14. A method of operation In a user input device com- 
5 prising a sensor capable of sensing surface relief of 

human skin, the method comprising: 

detecting movement of skin over the sensor; 
and 

10 altering the state of user interface apparatus in 

response to the detected movement. 
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